Systems and methods for the creation and playback of animated, interpretive, musical notation and audio synchronized with the recorded performance of an original artist

ABSTRACT

A music application guides a user with some musical experience through the steps of creating and editing a musical enhancement file that enhances and plays in synchronicity with an audio signal of an original artist&#39;s recorded performance. This enables others, perhaps with lesser musical ability than the original artist, to play-along with the original artist by following melodic, chordal, rhythmic, and verbal prompts. The music application accounts for differences in the timing of the performance from a standard tempo by guiding the user through the process of creating a tempo map for the performance and by associating the tempo map with MIDI information of the enhancement file. Enhancements may contain MIDI information, audio signal information, and/or video signal information which may be played back in synchronicity with the recorded performance to provide an aural and visual aid to others playing-along who may have less musical experience.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. provisional application No.60/412,458, filed Sep. 19, 2002, which is herebyincorporated-by-reference herein in its entirety.

BACKGROUND OF THE INVENTION

This invention relates to educational applications and, moreparticularly, to musical educational applications for “playing-along”with a recorded musical performance. This promotes the enjoyment andappreciation of the recorded musical performance.

Utility software that allows users to create new musical works whileincorporating previously recorded performances is well known.Illustrative commercially available utility software is provided, forexample, by Sonar™ 2002 by Twelve Tone Systems, Inc. of 51 MelcherStreet, Boston, Mass. Suitable hardware that may be used in implementingutility software such as Sonar™ 2002 may include a personal computer.This utility software allows a user to store and edit both MusicalInstrument Digital Interface (“MIDI”) commands and audio signals inorder to create a single new musical work.

MIDI is a specification and protocol that can be used to communicatenote and effect information between an electronic instrument and apersonal computer loaded with the utility software described above, forexample. An electronic instrument may simply be, for example, thepersonal computer's keyboard when the keyboard's buttons are assigned toparticular musical notes. The basic MIDI command recorded by the utilitysoftware is a “note on/off” event which comprises a note number (pitch)and a key velocity (loudness). There are various other types of valuesthat may be included in such a command, such as lyric text informationand instrument specific events. No audible sound data is actuallycontained in the MIDI commands. Although it may be displayed visually(as described below), in order to hear the MIDI information, the user'scomputer must be equipped with a MIDI output device such as a keyboard'ssynthesizer, a sound card with or coupled to a built in synthesizer, astand alone MIDI synthesis module, a “drum machine,” MIDI software, or acombination thereof, for example.

Audio signals (i.e., “sounds”), on the other hand, are produced when avibrating object excites air molecules in such a way to form coherentwaves or a series of pressure gradients which may impinge on theeardrum. Using an audio signal musical input device, such as atransducer (i.e., a microphone), analog audio signals (or “sound waves”)may be sampled by the utility software and stored as digital informationon the computer. When this sort of information is stored in the user'scomputer, it may be referred to as digital audio. An example of an audiosignal that may be stored by the user with the utility softwaredescribed above is the performance of a singer. A microphone may convertthe air pressure changes caused by the vibrations of the singer's vocalchords into voltage changes. These voltage changes may then be sampledand stored by the utility software onto the user's computer. The utilitysoftware stores this musical (i.e., audio) performance of the singer asdigital audio rather than MIDI information. Moreover, previouslyrecorded digital audio may be transferred from a compact disc (“CD”) andstored by the utility software onto the user's computer through a“direct” line-in signal input device, for example, as opposed to amicrophone.

Therefore, utility software may act as an audio and MIDI sequencer,allowing a user to piece together a new musical work with both MIDIcomponent tracks and audio signal tracks (such as recorded vocals orguitar) that are both controlled by one internal clock source. Suchsoftware may provide various aural and visual representations of thetracks which better allow a user to edit and playback specific aspectsof his or her composition. For example, MIDI information may bedisplayed as notation in staff view, guitar fret view, or as an eventlist, while digital audio information may be viewed as a continuousgraph of the audio signal's amplitude over time. Furthermore, the MIDIinformation and digital audio information may be heard at variousvolumes, tempos, and pitches, for example.

However, because the tempo often varies during the real performance,there is no satisfactory application supported by such utility softwarethat provides tools to assist and guide a user through the steps ofcreating, editing, and playing back enhancements to a recordedperformance of an original artist in such a way that promotes theenjoyment and appreciation of the recorded performance. As a result,many people who have access to the type of utility software describedabove do not use it in their efforts to enjoy and learn how to play anoriginal artist's recorded performance.

Accordingly, it would be desirable to provide a music application thatguides a user through the steps of creating and editing visual and auralmusical enhancements that play in synchronicity with a recordedperformance of an original artist, and that prompts the user to“play-along” with the musical enhancements to promote the enjoyment andappreciation of the original artist's recorded performance withoutrequiring possession of the performance itself, but that allows forplayback in synchronicity with the recorded performance if he or shepossesses it.

SUMMARY OF THE INVENTION

It is therefore an object of this invention to provide a musicapplication that guides a user through the steps of creating and editingvisual and aural musical enhancements that play in synchronicity with arecorded performance of an original artist.

It is therefore also an object of this invention to provide such a musicapplication that prompts the user to “play-along” with the musicalenhancements to promote the enjoyment and appreciation of the originalartist's recorded without requiring possession of the performanceitself, but that allows for playback in synchronicity with the recordedperformance if he or she possesses it.

These and other objects of the invention are accomplished in accordancewith the principles of the present invention by providing a musicapplication that guides a user through the process for enhancing anaudio signal. More specifically, the music application guides the userthrough the steps of creating and editing a musical enhancement filehaving visual and aural enhancements that play in synchronicity with arecorded performance of an original artist, and for prompting a user to“play-along” with the musical enhancement file to promote the enjoymentand appreciation of the original artist's recorded performance withoutrequiring playback of the performance itself.

It is to be understood that a user who creates a musical enhancementfile may be more knowledgeable about the intricacies of its associatedrecorded performance than another user who may just “play-along” withthe enhancement file. The music application of the present invention isintended to be participatory in nature, by both the novice and theadvanced enthusiast, in both the creation and use of musical enhancementfiles that are in synchronicity with recorded performances of originalartists, as well as in the subsequent revisions of the files to addfurther details. It is also to be understood that the user may save acreated enhancement file including the recorded performance, copyrightlaws permitting. A user may load a rightfully owned copy of the recordedperformance into an already created enhancement file for thatperformance, or may play-along to the enhancements of the enhancementfile without the performance.

The systems and methods of the present invention may be implementedusing iBook technology, as described in commonly-assigned U.S. Pat. No.6,052,717, which is hereby incorporated by reference herein in itsentirety. IBook technology allows users to contribute original material(e.g., a musical work) and create derivations of the original material.The systems and methods of the present invention may also be implementedusing the interactive web collaboration systems and methods of Reynoldset al. U.S. patent application Ser. No. 10/439,143, filed May 14, 2003,which is hereby incorporated by reference herein in its entirety.Reynolds et al. provides a framework and a space referred to as a topicroom in which one or more individuals may collaborate on topics ofmutual interest (e.g., music). In addition, the musical enhancementfiles of the present invention may be published and distributed usingthe systems and methods of Reynolds et al. PCT Publication No. WO02/088909, filed Feb. 1, 2002, which is hereby incorporated by referenceherein in its entirety.

In a preferred embodiment, the music application guides a user throughthe process of creating, editing, and/or “playing-along” with a musicalenhancement file. A musical enhancement file may comprise aural and/orvisual enhancements to a recorded performance of an original artist, andmay be saved separately from the recorded performance or, if inagreement with copyright laws, bundled with the recorded performance.These enhancements may be the user's interpretations of the recordedperformance's melodic, lyrical, and/or chordal structure. Theseenhancements may also be any other related aspect or accompaniment tothe performance that the user desires to create. The music applicationmay run a musical enhancement file in order to playback and present thevisual and/or aural portions of its enhancements to the user in variousways and at various pitches and tempos. The musical enhancement file maybe played at the same time as the recorded performance with which it issynchronized or it may be played by itself. One thing that distinguishesthe presentation of the musical enhancements of the present inventionfrom that of other media, such as Karaoke, is the fact that all of thevarious types of enhancements may be presented together and temporallysynchronized with the original recorded performance.

These enhancements can be conventional MIDI information and/or digitalaudio and video information. When the music application providesplayback of a musical enhancement file, a user is provided witheducational opportunities to “play-along” with, experiment with, study,and appreciate the musical structure and nuances of not only therecorded performance but also its enhancements. Throughout thisapplication, the term “play-along” refers to the many ways in which auser may play an instrument, sing, or just visually follow along withone or more enhancements created with the music application of thepresent invention. For example, some of the enhancements may bedisplayed visually while the user listens to the synchronized originalrecorded performance. Alternatively, all of the enhancements may bedisplayed visually while the user only listens to a particularenhancement and not the recorded performance, for example.

Time-stable pitch variation of the enhancements and the recordedperformance in synch with one another may add significant poignancyand/or accessibility to the subject matter by enabling a user toplay-along with the performance in a different pitch more suitable tothe range of their voice or instrument. Furthermore, pitch-stable speedvariation of the enhancements and the recorded performance may enable auser to play-along with the performance at a different speed moresuitable to their abilities.

A preferred process of the music application of the present inventionallows a user, who may not be a musician but who is interested inplaying a song, to play-along with a musical enhancement file insynchronicity with the original artist's recorded performance of thatsong with which the user is familiar. For example, the application maydisplay the visual portions of the enhancements in the form of animatedsheet music. This animated sheet music display may have chording symbolsor tablature, a musical stave representing the melody, and a line oflyrical text stacked vertically on top of each other and highlighted bya time cursor which scrolls over them with typically two measuresdisplayed at a time (see, e.g., FIG. 5) along with an optional leadingmeasure and trailing measure. As the performance progresses, themeasures may shift to the left so that the cursor may remain in the twocentral measures displayed, while the cursor continues to highlight theenhancements in synchronicity with the performance which may be aurallypresented to the user. Alternatively, the cursor may remain fixed,substantially to left-most portion of the display, while the visualportions of the enhancements continuously scroll through the cursor fromright to left, for example. Moreover, there may be two rows of theseselected enhancements stacked vertically, so that once the cursor hasscrolled to the end of the top row, it may jump down and begin to scrollthrough the bottom row while the top row is replaced with the next rowof enhancements that follows the row presently being scrolled. With thisenhancement file, and the ability to manipulate the display and playbackof the enhancements, users may be prompted to play-along while not onlylistening to but also “seeing” the music or other notations representingvarious intricacies of their favorite songs as performed by theirfavorite artists.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other advantages of the invention will be more apparentupon consideration of the following detailed description, taken inconjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 shows a diagram of a system suitable for implementation of amusic application in accordance with the present invention;

FIGS. 2 and 3 together form a flow chart of exemplary processes of themusic application in accordance with the present invention;

FIGS. 4 and 5 together form an exemplary display screen of a graphicaluser interface visual display of the music application in accordancewith the present invention; and

FIG. 6 is a more streamlined display screen of a graphical userinterface visual display of the application in accordance with thepresent invention suitable for playing-along to by a novice user.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 illustrates a musical system 100 suitable for implementation of amusic application in accordance with the present invention. System 100may include user computing equipment 110 that is connected by one ormore communication links 102 to a computer network 150. System 100 mayinclude music server 160 that is connected by one or more communicationlinks 104 to computer network 150.

The music application, implemented on system 100, may guide a userthrough enhancing a performance signal. In one suitable approach, themusic application may run on computing equipment. Computing equipment,such as computing equipment 110, may be a computer, personal computer,laptop computer, handheld computer, personal digital assistant, computerterminal, any other suitable computing device, or a combination of suchdevices. Computing equipment 110 may include memory and storage device120 in which utility software and data files of the music applicationmay be loaded and saved. Memory and storage device 120 may havecommunications, memory, and processing circuitry suitable for supportingfunctions such as accessing the music application over links 102.

In another suitable approach, the music application may run using aclient-server or distributed architecture where a portion of the musicapplication may be implemented locally on the computing equipment in theform of, for example, a client process. Another portion of the musicapplication may be implemented at a remote location, such as on musicserver 160 or any other suitable equipment as, for example, a serverprocess. A server such as music server 160 may be any suitable server,processor, computer, data processing device, or a combination of suchdevices. In one example, music server 160 may be a web server powered byFamily Systems' iBook System. These arrangements are merelyillustrative. Any other suitable arrangement for implementing the musicapplication may be used.

A user may interact with computing equipment 110 for selectingoperations and manipulating text and graphics of the music application'sgraphical user interface (“GUI”) using any suitable user input device130, such as a keyboard, a wireless keyboard, a mouse, a trackball, atouch pad, or any other suitable input device. User input device 130 maybe connected by one or more communication links 108 to computingequipment 110.

A user may input any musical information to computing equipment 110 forstoring recorded performances and/or creating musical enhancements usingmusical input device 140, which may be any MIDI-enabled instrument, suchas a guitar or keyboard synthesizer, a microphone, a CD player, adigital video disc (“DVD”) player, video camera, or any other suitablemusical input device for inputting MIDI information and audio and/orvideo signal information for creating enhancements and for loadingperformance signals. Musical input device 140 may be connected by one ormore communication links 112 to computing equipment 110.

MIDI information and audio signal information associated with the musicapplication may be played back as audible sound to a user using an auralspeaker device 170 that is connected by one or more communication links114 to computing equipment 110 that is equipped with a MIDI enabledsound card. Aural speaker device 170 may be a pair of stereo speakers,headphones, or any other suitable aural speaker equipment.

Text, graphics, and video signal information associated with the musicapplication may be presented to a user using a visual display device 180that is connected by one or more communication links 106 to computingequipment 110. Visual display device 180 may be a computer monitor, ahandheld computer display, a laptop computer display, a personal digitalassistant display, or any other suitable display device.

Computer network 150 may be a hub linking computing equipment 110 andmusic server 160. Music server 160 may be connected to computer network150 by one or more communication links 104. Computer network 150 may beany suitable computer network including the Internet, an Intranet, awide-area network (WAN), a local-area network (LAN), a wireless network,a digital subscriber line (DSL) network, a frame relay network, anasynchronous transfer mode (ATM) network, a virtual private network(VPN), or a combination of such networks.

The components of system 100 may be interconnected using variouscommunications links, such as links 102, 104, 106, 108, 112, and 114.These communications links may include satellite paths, coaxial cablepaths, fiber-optic paths, twisted pair paths, other wire or cable-basedlinks, modems, wireless paths through free space, any other suitablepaths, or a combination of such paths. Communications overcommunications links such as links 102, 104, 106, 108, 112, and 114 mayinvolve analog transmissions, digital transmissions, wirelesstransmissions, microwave transmissions, radio-frequency transmissions,optical transmissions, audio transmissions, any other suitable type oftransmissions, or a combination of such transmissions. Communicationsmay involve Internet transmissions, private network transmissions,packet-based transmissions, television channel transmissions,transmissions in the vertical blanking interval (VBI) of a televisionchannel or on a television sideband, MPEG transmissions, MIDIinformation transmissions, or any other suitable type of transmissions.Communications may involve wireless pager or other messagingtransmissions. These communications links may include cable connected tocable modems, digital subscriber lines, integrated services digitalnetwork (ISDN) lines, or any other suitable paths.

Users at user computing equipment 110 may participate in enhancing aperformance signal by interacting with the music application. The musicapplication may provide a user at computing equipment 110 with screenscontaining various application information on visual display device 180.For example, the user may be presented with screens that allow the userto create musical enhancements, view, edit, and play-along withenhancement information, or any other suitable screen in connection withthe application.

The music application may provide the user with interactive screenscontaining menus and selectable options that allow the user to navigatethrough the music application and participate in the music application.With user computing equipment 110, the user may use a keyboard, mouse,trackball, touch pad, or other suitable input or pointing device tonavigate the various menus and selectable options.

The user may access the music application with user computing equipment110. The user may access the music application by browsing to anInternet web site or a site on a private network, by running a localprogram, or by any other suitable method.

A music application for creating, editing, and playing-along with amusical enhancement file having visual and aural enhancements that playin synchronicity with a recorded performance and methods of use areprovided. Users may be individual musicians, conductors, symphonies,directors, or any other suitable entity that desires to enhance aperformance.

FIGS. 2 and 3 together show a flow chart 200 of exemplary processes ofthe music application of the present invention applied to a system suchas musical system 100 of FIG. 1 for guiding a user to create, edit, andplay-along with a musical enhancement file of an audio signal of anoriginal artist's recorded performance. Some or all of the steps may beperformed.

If the user desires to create a new musical enhancement file, the usermay begin the creation process at step 205 (e.g., by clicking on an iconlabeled “create new enhancement file”). The music application may thenprompt the user at step 210 to select and load the audio signal of anoriginal artist's recorded performance on which the new enhancement fileis to be based. This audio signal may be loaded from any compatibleaudio source, such as a CD, DVD, computer hard drive, or web site, forexample, in any suitable format, such as windows wave (“WAV”) or movingpicture experts group layer-3 (“MP3”), for example, via musical inputdevice 140 or music server 160 via computer network 150 (FIG. 1). Atstep 215, the music application may modify and/or compress/decompressthe contents of the loaded audio signal into a standardized file format(e.g., WAV or MP3) in order to optimize the suitability of the digitalaudio information for further processes, if necessary. Then, at step220, the music application may store the recorded performance as anoptimized digital audio track.

After the music application has loaded and modified/compressed therecorded performance's audio signal, it may prompt the user to generatea musical enhancement file at step 225. The musical enhancement file maybe based on current MIDI specifications and include the digital audiotrack of the already loaded and modified audio signal of the recordedperformance. The enhancement file may also contain additional data tooptimize its suitability for further processes, such as providing aframework for storing MIDI information and audio/video signalinformation (e.g., video of a musician playing-along with the recordedperformance) in additional tracks that are linked to the performancesignal track.

At step 230, the music application may prompt the user whether tomanually or automatically create a “timing track.” A timing trackintegrates the recorded performance's tempo, which may vary throughoutthe course of the song, with an internal clock source of computingequipment 110 (FIG. 1). If the user chooses for the timing track to becreated automatically (e.g., by clicking on an appropriate icon) at step230, the music application may generate the timing track by associatingcertain peaks of the recorded performance's audio signal with itsinherent sample timing, for example, at step 235. Alternatively, if theuser chooses to manually create the timing track at step 230, the musicapplication may prompt the user at step 240 to rhythmically tap outbeats in accordance with the tempo of the recorded performance's audiosignal as it progresses from beginning to end. This may be accomplishedin a plurality of ways, some of which may require the user to tap outbeats using a computer keyboard or mouse via user input device 130 or aninstrument via musical input device 140, for example. The musicapplication can discern the beats tapped out by the user and associatethem with a specific sample number within the related recordedperformance's audio signal, or it can directly integrate the beats intothe appropriate MIDI data bits.

At step 245, the music application may prompt the user to approve, edit,or redo the created timing track. If the user chooses to redo the timingtrack at step 245, the music application returns to step 230. If theuser chooses to edit the timing track at step 245, a number of variousediting options are provided by the music application at step 250,including playback of both the aural and visual portions of the recordedperformance's audio signal in synchronicity with the newly createdtiming track to assist the user in any necessary further refinement orcorrection. Editing of the timing track may be done by manual orautomatic means until the user and/or the music application is satisfiedthat valid data has been created. For example, upon playback, certaindeviations from the tempo of the artist's recorded performance may bediscerned in the timing track. In this case, the user may wish to editthese imperfections manually by “sliding” the position of a beat with amouse or by entering a specific time code position, for example.Alternatively, the music application may automatically discerninconsistencies in the timing track and automatically quantize thefrequency of the beats, for example. Once the user is satisfied with theintegrity of the timing track and chooses to approve it at step 245, themusic application stores the timing track in the musical enhancementfile at step 255.

Next, at step 260, the music application may prompt the user whether tomanually or automatically create a “MIDI tempo map” for the beats of thetiming track. A tempo map may appropriately vary the relationshipbetween the music application's MIDI information (e.g., the MIDIinformation of the timing track) and the music application's digitalsignal information (e.g., the signal information of the recordedperformance) to maintain the beat and measure boundaries of MIDIinformation tracks in synchronicity with the boundaries of the originalrecorded performance, for example. Measure boundaries may be assignedusing conventional musical notation, such as time signature, to specifythe number of beats per measure and the note value of each beat in thetiming track. The tempo map may assign multiple time signaturesthroughout the timing track in view of the fact that meter, in additionto tempo, may vary throughout the course of a musical performance, andthus throughout the timing track. If the user chooses for the tempo mapto be created automatically (e.g., by clicking on an appropriate icon)at step 260, the music application may generate the tempo map byassigning one or more time signatures to the timing track, and thus themusical enhancement file, for example, at step 265. Alternatively, ifthe user chooses to manually create the tempo map (e.g., by clicking onan appropriate icon) at step 260, the music application may prompt theuser, at step 270, to assign one or more time signatures to the timingtrack such that certain numbers of beats of the timing track and theirtime values in a measure may define a specific rhythm of the recordedperformance, for example.

At step 275, the music application may prompt the user to edit thecreated timing track or to approve, edit, or redo the created tempo map.If the user chooses to edit the timing track at step 275, the musicapplication returns to step 250. If the user chooses to redo the tempomap at step 275, the music application returns to step 260. If the userchooses to edit the tempo map at step 275, a number of various editingoptions may be provided by the music application at step 280, includingplayback of both the aural and visual portions of the recordedperformance's audio signal in synchronicity with the timing track andnewly created tempo map to assist the user in any necessary furtherrefinement or correction. Editing of the tempo map may be done by manualor automatic means until the user and/or the application is satisfiedthat valid data has been created. Once the user is satisfied with theintegrity of the timing track and its tempo map, and chooses to approvethe tempo map at step 275, the music application may store the tempo mapas proprietary data in the musical enhancement file at step 285.

Next, at step 300, the music application may prompt the user to create anew musical enhancement, edit an existing musical enhancement, or exitthe creation process. If the user indeed chooses to create a new musicalenhancement at step 300 (e.g., by clicking on an appropriately labeledicon), the music application may prompt the user, at step 305, to choosethe type of enhancement to be created.

The types of musical enhancements that the music application offers theuser to create at step 305 may include, but are not limited to, melodic,chordal, lyrical, rhythmical, instrumental, etc. In a preferredembodiment, the music application may suggest a particular order thatthe user should follow when deciding which type of enhancement to createnext at step 305. These enhancements may be created using user inputdevices 130, musical input devices 140, music servers 160 via computernetwork 150 (FIG. 1), or any of the other resources that may beavailable to the music application. Enhancements containing MIDIinformation are created in synchronicity with the recorded performance'saudio signal by being temporally linked with the MIDI information of thetempo map, which may serve as the rhythmic framework for the musicalenhancements and the enhancement file as a whole. Enhancementscontaining digital audio and/or video signal information are created insynchronicity with the recorded performance's audio signal by beingtemporally linked with the performance via time codes.

Once the user decides, at step 305, which type of enhancement is to becreated, the music application may prompt the user depending on the typeof enhancement chosen. As it is preferred in a particular embodiment, ifthe user first chooses to create a melodic enhancement at step 305(e.g., by clicking on an appropriately labeled icon), the musicapplication may prompt the user at step 310 to input MIDI informationthat is associated with a melody of the recorded performance's audiosignal and that is temporally associated with the tempo map. There aremany ways in which this may be facilitated by the music application. Forexample, the music application may playback the digital audio track ofthe recorded performance so that the user may play-along with a MIDIinstrument via musical input device 140 (FIG. 1). Alternatively, theuser may play-along by typing MIDI commands with a keyboard via userinput device 130 (FIG. 1) into a musical staff that is temporallyassociated with the tempo map and displayed by visual display device 180(FIG. 1), for example.

At step 330, the music application may then prompt the user to approve,edit, or redo the musical enhancement (e.g., the melodic enhancement, ifthe user created a melodic enhancement). If the user chooses to redo theenhancement at step 330, the music application may return to step 305.If the user chooses to edit the enhancement at step 330, various editingfacilities (described in more detail below) may be provided by the musicapplication at step 340 which allow the user to correct or refine theenhancement. For example, specialized aural playback facilities of themusic application, via aural speaker devices 170 (FIG. 1), allowrepetitive playback and/or playback at reduced tempos of selectedsegments of the enhancement and/or the temporally linked recordedperformance's audio signal as a means to expedite refinement orcorrection of the enhancement. Furthermore, specialized visual playbackfacilities of the music application, via visual display device 180 (FIG.1), allow the enhancement to be displayed during playback in differentways, such as in a musical staff or event list, for example. Once theuser is satisfied with the enhancement and chooses to approve it at step330, the music application may store the enhancement as a newenhancement track in the musical enhancement file at step 345, and maythen return to step 300.

Alternatively, or in addition to creating a melodic enhancement, theuser can create a lyrical enhancement at step 305 (e.g., by clicking onan appropriately labeled icon). If the user chooses to create a lyricalenhancement, the music application may prompt the user at step 315 toinput text information that is associated with lyrics of the recordedperformance's audio signal and whose syllables are temporally associatedwith notes of the melodic enhancement (if one has already been created).There are many ways in which this may be facilitated by the musicapplication. For example, the music application may prompt the user totype text information with a keyboard via user input device 130 (FIG. 1)into a text entry field displayed by video signal output device 180(FIG. 1), or load a text file from a lyric web site via computer network160 (FIG. 1), and then to assign each syllable in the entered text to anote or notes of the melodic enhancement. If no melodic enhancementexists, the music application may prompt the user to tap out melodicbeats, via devices 130 and/or 140 (FIG. 1), for example, in accordancewith the sounding of each of the lyric's syllables in the recordedperformance's audio signal as it progresses from beginning to end, andthen to assign each syllable in the entered text to those beats. Themusic application may also provide the user with facilities for theparsing of the text to assist in linking its syllabic content to eventsof the melodic enhancement or to tapped out melodic beats.

At step 330, the music application may then prompt the user to approve,edit, or redo the musical enhancement (e.g., the lyrical enhancement, ifthe user created a lyrical enhancement). If the user chooses to redo theenhancement at step 330, the music application may return to step 305.If the user chooses to edit the enhancement at step 330, various editingfacilities (described in more detail below) may be provided by the musicapplication at step 340 which allow the user to correct or refine theenhancement. For example, aural playback facilities of the musicapplication, via aural speaker devices 170 (FIG. 1), allow repetitiveplayback and/or playback at reduced tempos of selected segments of theenhancement and/or the temporally linked melodic enhancement and/or thetemporally linked recorded performance's audio signal as a means toexpedite refinement or correction of the enhancement. Furthermore,visual playback facilities of the music application, via visual displaydevices 180 (FIG. 1), allow the enhancement to be displayed duringplayback in different ways, such as in a string of syllables that aresynchronously highlighted and vertically aligned with the selected noteor notes of the melodic enhancement, for example. Once the user issatisfied with the enhancement and chooses to approve it at step 330,the music application may store the enhancement as a new enhancementtrack in the musical enhancement file at step 345, and may then returnto step 300.

Alternatively, or in addition to creating other enhancements, the usercan create a chordal enhancement at step 305 (e.g., by clicking on anappropriately labeled icon). If the user chooses to create a chordalenhancement, the music application may prompt the user at step 320 toinput MIDI information that is associated with a harmony or chording ofthe recorded performance's audio signal and that is temporallyassociated with the tempo map and notes of the melodic enhancement (ifone has already been created). There are many ways in which this may befacilitated by the music application. For example, the music applicationmay playback the digital audio track of the recorded performance so thatthe user may play-along with a MIDI instrument via musical input device140 (FIG. 1). Alternatively, the user may play-along by typing MIDIcommands with a keyboard via user input device 130 (FIG. 1) into amusical staff that is temporally associated with the tempo map anddisplayed by visual display device 180 (FIG. 1). The music applicationmay also provide the user with conventional notation facilitiesincluding, but not limited to, chord shape blocks, tablature (“tab”), oralphanumeric presentations. These notations may be retrieved by themusic application from a database of chord information, via storagedevice 120 (FIG. 1) or a web site via computer network 150 (FIG. 1), forexample, and may be placed in the enhancement's visual presentationautomatically or via user input device 130 (FIG. 1), for example. Themusic application may also allow modifications or addenda to thedatabase, which may be input via user input device 130 (FIG. 1) or by aninstrument via musical input device 140 (FIG. 1), for example, at theuser's discretion. For instance, rhythmic symbols may be added to thedisplay capabilities of this enhancement to show a particular strummingtechnique, emphasis, and/or sustain for each chord.

At step 330, the music application may then prompt the user to approve,edit, or redo the musical enhancement (e.g., the chordal enhancement, ifthe user created a chordal enhancement). If the user chooses to redo theenhancement at step 330, the music application may return to step 305.If the user chooses to edit the enhancement at step 330, various editingfacilities (described in more detail below) may be provided by the musicapplication at step 340 which allow the user to correct or refine theenhancement. For example, aural playback facilities of the musicapplication, via aural speaker devices 170 (FIG. 1), allow repetitiveplayback and/or playback at reduced tempos of selected segments of theenhancement and/or the temporally linked recorded performance's audiosignal and/or the temporally linked melodic enhancement (if one hasalready been created) as a means to expedite refinement or correction ofthe enhancement. Furthermore, visual playback facilities of the musicapplication, via visual display device 180 (FIG. 1), allow theenhancement to be displayed during playback in different ways, such asin a string of chord notations that are synchronously highlighted andvertically aligned with the correct note or beat in the melodicenhancement or tempo map, for example. Once the user is satisfied withthe enhancement and chooses to approve it at step 330, the musicapplication may store the enhancement as a new enhancement track in themusical enhancement file at step 345, and may then return to step 300.

Alternatively, or in addition to creating the other enhancements, theuser can choose to create one of the many other types of enhancementsnot already mentioned at step 305 (e.g., by clicking on an appropriatelylabeled icon). If the user chooses to create an other type of MIDIinformation enhancement, the music application may prompt the user atstep 325 to input, via musical input device 140 or computer network 150(FIG. 1), for example, MIDI information that is associated with anyinstrumentation, such as drums or bass, or any other type of musicalinformation that is a part of, or a creative addendum to, the recordedperformance's audio signal and that is temporally associated with thetempo map. As described above, at step 330, the music application maythen prompt the user to approve, edit, or redo the newly created musicalenhancement. Once the user is satisfied with the enhancement and choosesto approve it at step 330, the music application may store the MIDIenhancement as a new enhancement track in the musical enhancement fileat step 345, and may then return to step 300.

Alternatively, or in addition to creating an other type of MIDIinformation enhancement, the user may be prompted at step 325 to inputaudio and/or video signal information that is synchronized with therecorded performance's audio signal, via musical input device 140 orcomputer network 150 (FIG. 1), for example. Video representations ofguitar or piano fingering and/or audio representations of alternateaccompaniment materials are among the enhancements possible in thisapproach. Again, the user may be prompted at step 330 to approve, edit,or redo the newly created musical enhancement. Once the user issatisfied with the enhancement and chooses to approve it at step 330,the music application may store the audio and/or video signalenhancement as a new enhancement track in the musical enhancement fileat step 345, and may then return to step 300.

It is to be understood that any of the enhancement tracks may be createdautomatically by the music application through analysis of the recordedperformance's signal, for example. It is also to be understood that eachenhancement may be saved in one or more enhancement file and that anenhancement file may contain links to other enhancement files and otherenhancements, for example.

When the user is done creating all the desired enhancements, which maycontain standard MIDI components, audio and/or video signals, as well asother proprietary data, and the user chooses to exit the enhancementcreation process at step 300 (e.g., by clicking on an icon labeled“exit”), the music application may save the musical enhancement file(including the timing track, tempo map, any existing enhancement tracks,and the digital audio track of the recorded performance itself ifdesired and in agreement with copyright laws) at step 350 and thenreturn to step 205. The musical enhancement file may include all theaural and visual enhancements created by the user or time-coded linkagesthereto and is sufficient to reconstitute the enhancements when loadedby a user in the music application, whether or not the user, or the fileitself, is in possession of the audio signal of the recorded performancefor which the musical enhancement file was created to enhance. When itis desired to use the music application without the recordedperformance's audio signal, the enhancements may act as animated sheetmusic that does not require page turning and that may prompt the user toplay-along at a chosen tempo and pitch, for example. When used with therecorded performance's audio signal, not only may the user be promptedto play-along at a chosen tempo and pitch with this visual and auralanimated sheet music, but he or she is also able to hear insynchronicity the original performance played at the chosen tempo andpitch. The music application may enable and encourage the creator of amusical enhancement file to add his or her work to a library of worksthat may, for example, be located on a website using automated webpublishing and iBook technology as referred to above.

When the user desires to edit and/or playback an existing musicalenhancement file, the user may select the appropriate button or optionfrom the music application. For example, the music application, at step205, may also prompt the user to load an existing musical enhancementfile (e.g., by clicking on an icon labeled “load enhancement file”). Themusic application may then prompt the user at step 355 (FIG. 2) toselect and load a specific musical enhancement file from those availableon a local storage device or a remote web site via computer network 150(FIG. 1), for example. After the musical enhancement file has beenloaded, preferably by means of an auto-load function, the musicapplication may prompt the user at step 360 (FIG. 2) to load the audiosignal of the recorded performance that the loaded file enhances. Theuser may choose to complete this step if the user rightfully possessesthe correct audio signal and desires for it to be played-back insynchronicity with the enhancement file. The loading of the recordedperformance may be accomplished in a manner similar to step 210described above. Alternatively, step 360 (FIG. 2) may be completedautomatically by the music application. After step 360 (FIG. 2), themusic application may initialize a verification routine at step 365 toassess the contents of the loaded audio signal. This verification may beaccomplished using established CD database referral techniques or othersuitable methods, the results of which may render the recordedperformance's audio signal, title, and other linked informationavailable for the user to use with the loaded musical enhancement file.If the user does not rightfully possess the correct audio signal, themusic application may prompt the user with an offer to purchase it, forexample, through an affiliated web site via computer network 150 andmusic server 160 (FIG. 1). If the audio signal is purchased in thismanner, the music application may load the audio signal and then combineit with the enhancement file to make it ready for playback insynchronicity with the enhancements. Furthermore, the music applicationmay modify the audio signal to include a unique reference that mayidentify the original purchaser in case of an abuse of the performance'scopyright. In addition, if necessary, the music application may modifyand/or compress/decompress the recorded performance's audio signal atstep 370 (FIG. 2) in order to optimize its suitability for furtherprocesses and synchronize the audio signal with the enhancements of theloaded musical enhancement file, preferably automatically and similarly(at least in part) to step 215.

Next, at step 375 (FIG. 2), the music application may prompt the usereither to playback the enhancement file or to edit it. If the userchooses to edit the enhancement file at step 375, the music applicationmay return to step 275, from which point the user may choose to edit anyportion of the enhancement file. If the user chooses to playback theenhancement file at step 375 (FIG. 2), a number of various options maybe presented at step 380 (FIG. 2) which allow the user to choose in whatway or ways any or all of the enhancements are to be presented andplayed-back in order to meet the user's needs (see, e.g., FIGS. 4/5 and6). For example, the visual data of a melodic enhancement may bedisplayed by the music application as notes on a musical staff (e.g.,visual pane 475 of FIGS. 4/5 and 6) and/or as piano keys that are struckin accordance with the melody line, either in a user-configurable windowor multiple windows of the application's GUI via visual display device180 (FIG. 1). At the same time, the aural data of this melodicenhancement may be emitted through the music application's aural speakerdevice 170 (FIG. 1), for example.

As referred to above, and as illustrated in FIGS. 4 and 5 by anexemplary display screen 400 of the GUI of the music application whichmay be displayed via visual display device 180 (FIG. 1), the user may beprovided with a number of various tools to assist in the playback and/orediting of the enhancement file.

Display screen 400 may include control tool 405, much like those foundon common tape player transports, which provides play, rewind,fast-forward, and other functions to facilitate playback positioning forefficient use in editing and/or playing-along to the enhancement file.This may allow the user to quickly jump to, replay, or play-along withfavorite portions of a performance and its enhancements.

Display screen 400 may also include looping tool 410 that allows theuser to automatically repeat a user-selectable range of the performanceand/or its enhancement file. This may render a particularly difficultportion of a recorded performance easier to understand by allowing theuser to repeatedly hear and/or see it and its enhancements for thepurpose of helping him or her attain performance proficiency, forexample.

Display screen 400 may also include output tool 415 that allows the userto increase or decrease the tempo of the musical enhancement file andthe recorded performance's audio signal during playback, while at thesame time maintaining their original pitch, to facilitate user-analysisof difficult passages or to help the user create new enhancements. Byplaying-along with a recorded performance and enhancements thatgradually speed up, the user may perfect his or her personalperformance. Output tool 415 may also allow the user to increase ordecrease the pitch of a musical enhancement file and the recordedperformance's audio signal during playback, while at the same timemaintaining their original tempo. This may enable the user to explorethe recorded performance's audio signal in a range more suitable for hisor her voice or instrument. Output tool 415 may also provide the userwith the ability to adjust various aural characteristics of the overallenhancement file, such as volume, balance, and frequency makeup (e.g.,“bass”, “mid-range”, and “treble” frequencies), for example, duringplayback.

Separate windows may be provided by the music application and displayedsuch that the user is able to independently control the aural and visualpresentation of the recorded performance and each of its enhancements.For example, recorded performance window 420 may be provided with auralcontrol tool 425, visual control tool 430, and visual pane 435. Auralcontrol tool 425 may include controls similar to those of output tool415 which allow adjustment to various aural characteristics of therecorded performance's audio signal, such as volume, balance, andfrequency makeup for example, during playback of the enhancement file.Aural control tool 425 may also include controls which allow the user tomute or solo the performance during playback of the enhancement file.Moreover, aural control tool 425 may include controls which allow theuser to choose the output device through which the performance is to beplayed back. Visual control tool 430 may include controls which allowthe user to choose in which way or ways visual pane 435 displays aspectsof the recorded performance. For example, and as illustrated, visualcontrol tool 430 may be adjusted such that visual pane 435 shows acontinuous waveform of the performance's amplitude over time.Alternatively or additionally, visual pane 435 may show the makeup ofthe performance's frequency spectrum, for example. Visual control tool430 may also provide the user with the ability to adjust the amount ofinformation illustrated by visual pane 435 and the ability to adjust thesize and shape of pane 435 and recorded performance window 420, itself.

Timing track window 440 may also be provided by the music application.Timing track window 440 may include aural control tool 445, visualcontrol tool 450, and visual pane 455. Aural control tool 445 mayinclude controls which allow adjustment to various aural characteristicsof the timing track's MIDI information, such as volume, balance,velocity, and reverb, for example, during playback of the enhancementfile. Aural control tool 445 may also include controls which allow theuser to mute or solo the timing track and/or record additional MIDIinformation therein during playback of the enhancement file. Moreover,aural control tool 445 may include controls which allow the user tochoose the output device to which the timing track's MIDI information iscommunicated during playback. Visual control tool 450 may includecontrols which allow the user to choose in which way or ways visual pane455 displays aspects of the timing track. For example, and asillustrated, visual control tool 450 may be adjusted such that visualpane 455 shows a piano roll of the timing track's MIDI information.Alternatively or additionally, visual pane 455 may show an event list ofthe information, for example. Visual control tool 450 may also providethe user with the ability to adjust the amount of informationillustrated by visual pane 455 and the ability to adjust the size andshape of pane 455 and timing track window 440, itself.

Melodic enhancement window 460 may also be provided by the musicapplication. Melodic enhancement window 460 may include aural controltool 465, visual control tool 470, and visual pane 475. Aural controltool 465 may include controls, similar to those of aural control tool445, which allow adjustment to various aural characteristics of themelodic enhancement track's MIDI information during playback of theenhancement file. Aural control tool 465 may also include controls whichallow the user to mute or solo the melodic enhancement and/or recordadditional MIDI information therein during playback of the enhancementfile. Moreover, like aural control tool 445, aural control tool 465 mayinclude controls which allow the user to choose the output device towhich the melodic enhancement's MIDI information is communicated duringplayback. Visual control tool 470 may include controls, similar to thoseof visual control tools 430 and 450, which allow the user to choose inwhich way or ways visual pane 475 displays aspects of the melodicenhancement. For example, and as illustrated, visual control tool 470may be adjusted such that visual pane 475 shows a musical staff of themelodic enhancement's MIDI information. Alternatively or additionally,visual pane 475 may show a piano roll of the information, for example.Like tools 430 and 450, visual control tool 470 may also provide theuser with the ability to adjust the amount of information illustrated byvisual pane 475 and the ability to adjust the size and shape of pane 475and melodic enhancement window 460, itself.

Chordal enhancement window 480 may also be provided by the musicapplication. Chordal enhancement window 480 may include aural controltool 485, visual control tool 490, and visual pane 495. Aural controltool 485 may include controls, similar to those of aural control tools445 and 465, which allow adjustment to various aural characteristics ofthe chordal enhancement track's MIDI information during playback of theenhancement file. Aural control tool 485 may also include controls whichallow the user to mute or solo the chordal enhancement and/or recordadditional MIDI information therein during playback of the enhancementfile. Moreover, like aural control tools 445 and 465, aural control tool485 may include controls which allow the user to choose the outputdevice to which the chordal enhancement's MIDI information iscommunicated during playback. Visual control tool 490 may includecontrols, similar to those of visual control tools 430, 450, and 470,which allow the user to choose in which way or ways visual pane 495displays aspects of the chordal enhancement. For example, and asillustrated, visual control tool 490 may be adjusted such that visualpane 495 shows tablature notation for a 6-string guitar of the chordalenhancement's MIDI information. Alternatively or additionally, visualpane 495 may show a musical staff of the information, for example. Liketools 430, 450, and 470, visual control tool 490 may also provide theuser with the ability to adjust the amount of information illustrated byvisual pane 495 and the ability to adjust the size and shape of pane 495and chordal enhancement window 480, itself.

Lyrical enhancement window 500 may also be provided by the musicapplication. Lyrical enhancement window 500 may include visual controltool 505, and visual pane 510. Visual control tool 505 may includecontrols, similar to those of visual control tools 430, 450, 470, and490, which allow the user to choose in which way or ways visual pane 510displays aspects of the lyrical enhancement. For example, and asillustrated, visual control tool 505 may be adjusted such that visualpane 510 shows both a single-line text field for displaying the lyricaltext that is temporally linked to the portions of the other tracks thatare displayed in visual panes 435, 455, 475, and 495, for example, and amultiple-line text field for displaying one or more stanzas of thelyrical text while boxing the syllable being played back, for example.Alternatively, visual pane 510 may show just one parsed word of thelyrical text at a time. Visual control tool 505 may also includecontrols which allow adjustment to various visual characteristics of thelyrical enhancement track's text information during playback of theenhancement file, such as font, type size, and color, for example. Likevisual control tools 430, 450, 470, and 490, visual control tool 505 mayalso provide the user with the ability to adjust the amount ofinformation illustrated by visual pane 510 and the ability to adjust thesize and shape of pane 510 and lyrical enhancement window 500, itself.

Audio/video signal enhancement window 515 may also be provided by themusic application. Audio/video signal enhancement window 515 may includeaural control tool 520, visual control tool 525, and visual pane 530.Aural control tool 520 may include controls similar to those of auralcontrol tool 425, which allow adjustment to various auralcharacteristics of the audio/video signal during playback of theenhancement file. Aural control tool 520 may also include controls whichallow the user to mute or solo the audio/video signal enhancement and/orrecord additional audio and/or video signal information thereover duringplayback of the enhancement file. Moreover, like aural control tools445, 465, and 485, aural control tool 520 may include controls whichallow the user to choose the output device to which the audio/videoenhancement's audio signal information is communicated during playback.Visual control tool 525 may include controls, similar to those of visualcontrol tools 430, 450, 470, and 490, which allow the user to choose inwhich way or ways visual pane 530 displays aspects of the audio/videoenhancement. For example, and as illustrated, visual control tool 525may be adjusted such that visual pane 530 just shows the video signalinformation of the audio/video enhancement. Alternatively, instead ofdisplaying video signal information, visual pane 530 may just show acontinuous waveform of the amplitude of the audio signal of theaudio/video enhancement over time, for example. Visual control tool 525may also include controls which allow adjustment to various visualcharacteristics of the audio/video enhancement's video signalinformation during playback of the enhancement file, such as picturecolor, contrast, and resolution, for example. Like visual control tools430, 450, 470, and 490, visual control tool 525 may also provide theuser with the ability to adjust the amount of information illustrated byvisual pane 530 and the ability to adjust the size and shape of pane 530and audio/video signal enhancement window 515, itself.

Cursor window 535 may also be provided by the music application. Cursorwindow 535 may include visual control tool 540 and cursor 545.Preferably, cursor 545 sweeps horizontally across each of the verticallyaligned track windows 420, 440, 460, 480, and 500, while highlightingthe portion of information in its pane to be played-along at thatparticular moment during playback of the enhancement file. Visualcontrol tool 540 may include controls which allow the user to choose inwhich way or ways cursor 545 visually indicates which portion of each ofthe displayed file's tracks are to be played back at that particularmoment. For example, and as illustrated, visual control tool 540 may beadjusted such that cursor 545 simply crosses each of the panes of theenhancement file without altering the display of the information.Alternatively, cursor 545 may highlight in yellow the information ineach visual pane that is to be played back at that particular moment,for example.

Therefore, the music application is able to simultaneously presentmultiple temporally linked enhancements, while having each enhancement'saural data emitted at different volumes through different devices, suchas aural speaker devices 170 (FIG. 1), for example. These features ofthe music application of the present invention may allow a band ororchestra to utilize an enhancement file that contains enhancementtracks, such as animated sheet music, for each instrument found in thatband or orchestra, for example. The entire group may be able to viewvisual components of all the file's enhancements on one large display,while each member of the group may selectively listen only to the auralcomponent and view only the visual component of their particularinstrument's enhancement via headphones and a personal video screen,respectively.

The music application may also provide the user with a “zoom playback”tool which allows for customizable selection and sizing of displaywindows and their contents. For example, a user may wish to see amelodic enhancement for certain sections of the performance, and havethe chordal enhancement shown for other sections of the performanceduring playback of the enhancement file.

As yet another example, the music application may provide chord studytools to allow linking (e.g., via the web) of a chordal enhancement'sexisting information to a central database of chord shapes, fingerings,and/or alternate tunings. This database, which can be constantly updatedby various users as they discover different ways to play a particularchord, allows a particular user to choose amongst the countless ways inwhich a particular performance's chording may be accomplished. Forexample, depending upon the manner in which the user tunes his or herinstrument, a chordal enhancement's information may be adjustedautomatically or by visual control tool 490, for example, in order toshow the most simplified way to play each chord for that particulartuning. The same may be said depending on the particular instrument theuser plays. For example, if the chordal enhancement was created to showthe fingering for a guitar, it may be linked and adjusted automaticallyor by visual control tool 490, for example, so that the chordalinformation is displayed by the application for a mandolin, if that isthe instrument with which the user wishes to play-along.

Moreover, the music application may prompt the user to play-along withan enhancement file for inputting certain information. The musicapplication of the present invention may store this user-createdinformation and compare it to the information of the enhancements withwhich the user played-along. Then, tutorial data indicating correctlyand incorrectly played notes, for example, may be automaticallygenerated by the music application in order to teach the user how tobetter play-along with the enhancements, and therefore the recordedperformance.

FIG. 6 shows a more streamlined display screen 600 of the musicapplication of the present invention, which may be displayed via visualdisplay device 180 (FIG. 1) for a user who simply desires to play-alongwith a recorded performance by singing and/or playing an instrument(e.g., a guitar) while viewing only four measures at a time. The usermay configure the music application to display only basic control tool405, chordal enhancement track's visual pane 495 showing simplefingering for a guitarist, melodic enhancement track's visual pane 475showing a staff representation of the enhancement's MIDI information,and lyrical enhancement track's visual pane 510 showing only asingle-line text field of the enhancement's text information. Alsodisplayed is cursor window 535 and time cursor 545 that sweepshorizontally across the vertically aligned enhancement panes, whilehighlighting the chord, melodic note, and lyrical syllable to beplayed-along in synchronicity with the recorded performance at thatparticular moment during playback of the enhancement file. Thissimplified and abbreviated display 600 allows a user to concentratesolely on the musical enhancements that they wish to play insynchronicity with the recorded performance, without having to flip themany pages of contemporary sheet music or look in three very separateplaces for the chording, melody, and lyrics to a song.

The screens shown in FIGS. 4-6 are merely illustrative. Any othersuitable menus, options, control tools, visual panes, text boxes, orother content may be provided.

It may be understood that a significantly more basic music applicationin accordance with the principles of the present invention, whileyielding similar benefits to a user, may exist whereby after loading arecorded performance's audio signal, the music application may simplyprompt a user to load a scanned copy of the performance's conventionalsheet music into a “scanned picture” track of a new musical enhancementfile. The music application may then, preferably automatically, stretchor shrink this scanned picture of sheet music and sequence it so that itspans the entire duration of the recorded performance's audio signal.Next, time cursor 545 may be used to sweep horizontally across thescanned sheet music while highlighting the part of the sheet music thatcorresponds to the recorded performance's audio signal that may beplayed back simultaneously. While this embodiment of the applicationdoes not require the creation of MIDI data enhancements for variousinstruments and melodies of a performance, and may therefore be easierto use, sheet music is often not entirely accurate and may fail toinclude notes that are a part of the recorded performance being used.

While the music application of the present invention is described usingMIDI protocol for inputting and storing certain user enhancements, itshould be understood that any other suitable audio/visual format may beused. For example, it is to be understood that the processes describedabove and shown in FIGS. 2 and 3 are merely illustrative. Some or all ofthe steps may be performed depending on the type of performance beingenhanced, the types of enhancements a user creates, or the types ofenhancements existing in an already created enhancement file. Forexample, if an enhancement file is to be created including only audioand/or video signal enhancements to a recorded audio signal performance,a user may choose not to create a timing track and tempo map, and theaudio/video signal information for each enhancement may be linked to therecorded performance via time codes of the application.

Although the music application of the present invention is described toprompt a user to create, edit, and/or play-along with enhancements insynchronicity with a recorded performance of a familiar song recorded bya familiar artist, it has many other beneficial implementations. Forexample, a first user may create a new enhancement file that includesenhancements to an original performance that the first user alsocreated. This enhancement file may then be shared with a second userthrough a particular topic room of interactive web collaborationtechnology, as described in, for example, Reynolds et al. U.S. patentapplication Ser. No. 10/439,143, filed May 14, 2003. Then, the seconduser may add additional audio, video, or MIDI enhancement tracks, suchas another instrument or chordal enhancement, for example. These may bethe second user's musical interpretation or addition to the first user'senhancement file. In this way, each user may “jam-along” with the otherby creating musical enhancements that enhance not only the originalperformance but also each user's contributions to the performance.

Furthermore, while the music application of the present invention isdescribed as guiding a user through a process of creating, revising, orplaying-along with a musical enhancement file that may comprise auraland visual enhancements to a recorded audio performance, it should beunderstood that any suitable multimedia, such as an audio-visualperformance, may be enhanced by an enhancement file. For example,instead of a recorded audio performance, the music application of thepresent invention may load the signal(s) of a recorded audio-visualperformance (e.g., a live concert or play from a DVD) via musical inputdevice 140, for example. Then the music application may prompt a user tofollow the steps described above for enhancing not only the audio signalbut also the video signal of the recorded performance.

For example, if the recorded performance used is the video of atheatrical play, one particular enhancement that a user may desire tocreate could be an animation showing a floor plan of the stage thattracks the movement of the performers throughout the performance. Thiswould be particularly useful to a director who is interested in usingthe enhancement to teach the actors, individually or as a groupdepending on the manner in which the enhancement is displayed, where tobe on stage during a particular scene. Furthermore, various users,independently of one another, could provide their own interpretiveperformances of various characters in a specific play. The applicationcould then combine each performance in synchronicity with one another inorder to create a theatrical enhancement of that specific play, such asanimation, using the actors' voices to accompany it, for example.Furthermore, any phrase by any actor can be programmed to be triggeredby a triggering phrase by another actor, or by a combination or sequenceof such phrases by one or more actors.

Thus it is seen that a music application for guiding a user through thesteps of creating, editing, and playing back enhancements to a recordedperformance of an original artist has been provided. It should be notedthat the types of created enhancements, along with their sizes, formats,and quantity, as described above, are only exemplary. One skilled in theart will appreciate that the present invention can be practiced by otherthan the described embodiments, which are presented for purposes ofillustration and not of limitation, and the present invention is limitedonly by the claims which follow.

1-60. (canceled)
 61. A system for enabling a user to accompany arecorded performance, the system being configured to: synchronizeenhancement data with the recorded performance; store the synchronizedenhancement data; use the synchronized enhancement data to prompt theuser to play along with the recorded performance; receive input datafrom a musical input device; store the input data; compare the inputdata and the synchronized enhancement data; and generate tutorial databased on the comparison, wherein at least a portion of the tutorial dataindicates correctly and incorrectly played elements of the enhancementdata.
 62. The system of claim 61, wherein the system is furtherconfigured to synchronize the enhancement data to a time relative to thestart of the recorded performance.
 63. The system of claim 61, whereinthe system is further configured to: store information in accordancewith the tempo of the contents of the recorded performance in a tempotrack; and synchronize the enhancement data to a time relative to amusical beat of the tempo track.
 64. The system of claim 63, wherein thesystem is further configured to display a highlight region on a visualportion of the synchronized enhancement data based at least in part on amusical beat of the tempo track.
 65. The system of claim 64, wherein thesystem is further configured to scroll the highlight region along thevisual portion of the synchronized enhancement data.
 66. The system ofclaim 61, wherein the system is further configured to present thetutorial data to the user.
 67. The system of claim 61, wherein thesystem is further configured to use the synchronized enhancement data toprompt the user to play along with the recorded performance furthercomprises by displaying a visual portion of the synchronized enhancementdata.
 68. The system of claim 67, wherein the visual portion of thesynchronized enhancement data comprises at least one of chord tablature,chord shape, chord symbols, alphanumeric representations, or a visualrepresentation of chord fingering.
 69. The system of claim 67, whereinthe visual portion of the synchronized enhancement data comprises noteson at least one of a musical stave, piano roll, tablature, oralphanumeric representations that represent the melody of the recordedperformance.
 70. The system of claim 65, wherein the visual portion ofthe synchronized enhancement data is divided into measures.
 71. Thesystem of claim 65, wherein the system is further configured to scroll avisible portion of the visual portion of the synchronized enhancementdata.
 72. The system of claim 61, wherein the system is furtherconfigured to use the synchronized enhancement data to prompt the userto play along with the recorded performance further comprises by playingan aural portion of the synchronized enhancement data.
 73. The system ofclaim 61, wherein the system is further configured to use a time-stablepitch variation of the synchronized enhancement data to prompt the userto play-along with the recorded performance.
 74. The system of claim 63,wherein the pitch of the time-stable pitch variation is selected by theuser.
 75. The system of claim 63, wherein the system is furtherconfigured to apply a time-stable pitch variation to the recordedperformance that is the same as the time-stable pitch variation of thesynchronized enhancement data.
 76. The system of claim 61, wherein thesystem is further configured to use a pitch-stable speed variation ofthe synchronized enhancement data to prompt the user to play along withthe recorded performance.
 77. The system of claim 76, where the speed ofthe pitch-stable speed variation is selected by the user.
 78. The systemof claim 16, wherein the system is further configured to apply apitch-stable speed variation to the recorded performance that is thesame as the pitch-stable speed variation of the synchronized enhancementdata.
 79. The system of claim 61, wherein the system is furtherconfigured to provide the user with the opportunity to navigate to atleast one of a moment of the recorded performance or elementsynchronized enhancement data.
 80. The system of claim 79, wherein thesystem is further configured to repeat a portion of the synchronizedenhancement data at least one of before or after the at least one momentor element.
 81. The system of claim 80, wherein the system is furtherconfigured to repeat a portion of the recorded performance thatcorresponds to the repeated portion of the synchronized enhancementdata.
 82. The system of claim 61, wherein the synchronized enhancementdata comprises at least one chord representation associated with therecorded performance.
 83. The system of claim 82, wherein the system isfurther configured to adjust the at least one chord representation basedon at least one chord shape, chord fingering, alternative tuning, ormusical instrument.
 84. The system of claim 61, wherein the synchronizedenhancement data comprises lyrical data.
 85. The system of claim 84,wherein the lyrical data is audible.
 86. The system of claim 84, whereinthe lyrical data is visual.
 87. The system of claim 61, wherein thesynchronized enhancement data further comprises melodic data.
 88. Thesystem of claim 87, wherein the melodic data is audible.
 89. The systemof claim 87, wherein the melodic data is visual.
 90. The system of claim61, wherein the recorded performance is an audio signal.
 91. The systemof claim 61, wherein the recorded performance is an audio/video signal.92. The system of claim 61, wherein at least a portion of thesynchronized enhancement data comprises MIDI information.
 93. The systemof claim 61, wherein at least a portion of the synchronized enhancementdata comprises audio/video.
 94. A method for enabling a user toaccompany a recorded performance, the method comprising: synchronizingenhancement data with the recorded performance; storing the synchronizedenhancement data; using the synchronized enhancement data to prompt theuser to play along with the recorded performance; receiving input datafrom a musical input device; storing the input data; comparing the inputdata and the synchronized enhancement data; and generating tutorial databased on the comparison, wherein at least a portion of the tutorial dataindicates correctly and incorrectly played elements of the enhancementdata.